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ABSTRACT 

A sample of approximately 385 heari 
elementary and secondary school students whose math 
achievement ranged from second to sixth grades were 
determine, the effectiveness of a computer assisted 
mathematics program on the acquisition of computati 
students selected were from a population of over 20 
schools for the deaf in four states and the Distx;ic 
curriculum provided for individualized instruction, 
areas of proficiency, drill in areas of deficiency, 
in profile form on student progress. The curriculum 
the following 14 sequential mathematical strands: n 
horizontal addition, horizontal subtraction, vertic 
vertical subtraction, equations, measurement, horiz 
multiplication, laws of arithmetic, vertical multip 
division, fractions, decimals, and negative numbers 
77 students each were assigned 10, 30, 70, 100, and 
the computer. Major results of pretest and posttest 
that the computer assisted curriculum enabled heari 
students to achieve gains expected of normally hear 
greater numbers of sessions on the computer were be 
students, and that the gains could be achieved in s 
to 10 minutes per day. (MC) 
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EVALUATION OF COMPUTER-ASSISTED- INSTRUCTION IN ELEMENTARY MATHEMATICS 

FOR HEARING- IMPAIRED STUDENTS* 

1 

P, Suppes, J, D. Fletcher, M. Zanotti\ 
P* V. Lor ton, Jr., and B. W, Searle 

• ' INTRODUCTION 

The Institute for Mathematical Studies in the Social Sciences at 
Stanford University (IMS3S) has been developing curriculums and tech- 
niques for computer -assisted instruction (CA.I.) since 1963- • In 1970 ■ 
the Office of Education funded the Institute for -GA.I research in schools 
for the deaf. During the 1970-71 school year approximately 1,000 stu- 
dents in schools for the deaf participated in the project, and during 
1971-72 this number increased to more than 2,000 students' taking CAJ 
lessons at 15 schools for the deaf located in four states and the' 
District of Columbia, 

The students who participated in the experiment reported in this" 
article were chosen from among the entire population of deaf- students 
receiving CAI lessons in mathematics and language arts through the 
IMSSS system in 1971-72* The degree of hearing loss among the students 
"was essentially that adopted for admission standards by the schools „ 

Generally, this loss is at least 60 decibels in the better ear . Stu- 

• -.. \ • \ ■ 

dents selected -by the schoois for CAI represent a cross section- of 

• ■ ..... \ • 

*This research was supported by Office of Education Grant 
OEG-0-70-V797(607), 0E Project No, 1^-2280, 



their elementary and secondary school population. Some of .these students 
may 'be significantly handicapped in ways other than that- of hearing loss, 
but these additional handicaps do not prohibit the students from meeting 
primary school advancement requirements/ 

This experiment evaluated the effectiveness of the CAI mathematics 
program .on the- acquisition of computational skills. We first describe, 
the models of evaluation tested, and then the curriculum, before turning 
to .the analysis of data. In the models tested, we have emphasized . 
variation x ln\ A intensity of experimental treatment, as opposed to simply 
comparing experimental with control .groups. (" 

THE MODELS 

To investigate the relationship of posttreatment scores to pre- 
treatment scores and the number of CAI mathematics strands sessions 
given, five models were tes.ted, In all five models, is the pre- 

. treatment score of student i. T. rt is the posttreatment score of 

■ i2 

• student i, and IT is the number of CAI mathematics strands sessions 
taken by M student i.' Following standard notation, E(T 12 ) is the . 
expected posttreatment score of student i. 

Model I: Linear. . J . 

• • : EC v- a o + a i T ii + '¥i' 

. In this modeT, the expected, effect of pretreatment score and number of 
sessions oh posttreatment performance is assumed to be linear. 

Model II: Linear with interaction ; 

ER?C 2 



In Model II, a linear effect of pretreatment score and. number of ses- 
sions .is assumed, but a linear effect, from the interaction of pretreat- 
ment; score and sessions is also postulated. 

Model III: Cobb-Douglas,, 

E(in T.J = a n + a. in T.. + a n in N. . 
v 12' 0 1 il 2 i 

Model III is based on a formulation of the Cobb-Douglas type (from 

econometrics ) 3 namely, 

a l a 2 
T. c = a n T._K. . 
i2 0 il i 

This model is multiplicative and assumes a 'weighted interaction* between 
pretreatment score and number of sessions in accounting for change in 
posttreatment scores, 

Model IV: Log quadratic. 

E(T i2 ) - a Q + a 1 T il + a 2 in N. + a (in N.) 2 + a^(in N. ) 5 , 

In' Model IV, the effect of the pretreatment score is assumed to be linear 
but the effect of number of sessions is assumed to be logarithmic, rather 
than linear, In order to explore this logarithmic assumption fully, we . 
included second- and third- order terms in in 

Model V:. Exponential* 

S(Jn T. 2 ) = a Q + a lNi T n . 

Model V is based on an exponential formulation, namely, 

*i2 = a 0 e 

In this model, the effect of number of sessions and pretreatment score 
may he strictly increasing or strictly decreasing, depending on the 
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sign of Pretreatment score and number of sessions ore assumed 

to interact, 

In each of these models we treat pretreatment and post. treatment 
scores separately, l.e. ; we do not, for instance, regress the differ- 
ence in the two scores on the number of CA1 sessions. Even though 
^ reasons for avoiding difference scores have been discussed from several 
standpoints in the literature of evaluation (e,g,, Cronbach & -Pur by, 
1970 j Lord, 1963), we give here direct but elementary analysis from 
first principles. So far as we know, this argument has not appeared 
in this form in the literature. 

Let X, Y, and N be random variables with E(X) = E(Y) = E(N) = .0, 
and with finite, nonzero variance. The pairwise correlation coefficients, 
of these random variables are then well defined. Consider now the cor- 
relation 



r(Y - X,N) = 



E((Y - X.)N) 
cr(Y - X)cr(N) ' 



1 



From the linearity of the expectation operator, 



E((Y - X)W) = E(YN) - E(XN) , 



(2) 



Further, since by hypothesis E(X) = E(Y) = Q, 



a 2 (Y - X) = E((Y - X) 2 ) - E(X 2 ) + E(Y 2 ) 

■ E(X 2 ) + E(Y 2 ) 
= a 2 (X) + a 2 (Y) 



2E(.XY) 



2r(X,Y}a(X)a(Y) 



2r(X J Y)cr(x)a(Y) . 



(3) 



Substituting (2) and (3) into equation (l) we obtain: 



r(Y - X,N) = 



E(YK) - E(XN) 



a(N){a 2 (X) + a 2 (Y)'- 2a(x)a(Y)r (X, Y)) 1 / 2 ' 




Let us now assume that the correlation of X and N is ?,ero, i.e., 

rU,Vi) 0 ' (I) 

and also that X and Y are highly correlated,, but r(X,Y) <- 1, and 
that approximately 

cr(X) « r(X,X)ff(Y) , (II) 
then^ on the basis of (I) and (ll) 

r(Y - X,N) *■ p 5 

tr(N){a (Y) - a 2 (Y)r 2 (X,Y)}^ 2 

_ ■r(Y,K)a (Y)q(N) 



(r(N)<r(Y)(l - r 2 (X,x)) 1/2 ° 
Therefore ? 

r(Y - X,N) * ■ r(Y ? N) . (4) 

Equation (4) shows that with appropriate choice of a random variable 
X, r(X,Y) large ? and X orthogonal to we can obtain a correlation 

coefficient, r(Y - X P N)j as close to 1 as we please so long as r(Y/N) 
is not zero, The correlation of N with the Y - X difference score 
may depend solely on the correlation of Y with X and may have nothing 
to do with, the effect of N„ . For that matter,, 

r 2 (XN) + r 2 (X,Y) - 1 . 

implies that 

r(Y - X,N) - 1 . 



5 



'The geometrical i>.t crywotat:! on is ctloer. 




Y' 



Assumption (ll), <r(X) ~ r(X,Y)cr(Y), is the requirement that the norm 
of the projection Y' of Y on X approximately equals the norm of X. 

Assumptions (i) and (II) hold approximately for many experiments 
using Y as the posttreatment measure, X as the pretreatment mea- 
sure, and N as some measure of the amount or intensity of treatment, 

From these and other results in the literature about difference 
scores, it is clear that studies of evaluation that use models built 
around difference scores must be approached with considerable caution, 
We numerically illustrate later in the analysis of data pertinent to 
the five models these remarks about difference scores, even though for 
practical purposes we include some general remarks about gains in the 

analysis of data, 

/, 

\ 

i 

\ 

\ 
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THE MATHEMATICS STRANDS. CURRICULUM 



The research reported here attempts to assess the pedagogical effect, 
of the Institute's elementary ma thematic 5 curriculum on "achievement, amor:,' 
the hearing-impaired students, CAI in elementary mathematics provided- bv 
the Institute has had a history of success with 'hearing .students (Suppes 
Morningstar^ 19&9.? 1970a) , and comparable success was anticipated for stu 
dents in schools for the deaf, 

The objectives of the -strands program are (a) to provide supple- 
mentary individualized instruction in elementary mathematics at a level 
of difficulty appropriate to each student's level of achievement, (b) to 
allow acceleration in any concept area in which a student demonstrates 
proficiency, and to allow repeated drill and practice: in areas of de- 
ficiency, and (c) to report a daily profile of each student:/ s progress 
through the curriculum, 

A strand is a series' of exercises of the same logical type (e.g,, 
horizontal addition, vertical subtraction, multiplication of fractions) 
arranged sequentially in equivalence classes according to their relative 
difficulty, The Ik strand;? in the program and the grade levels spanned 
by each strand are shown in Table 1„ Each strand contains either five 

Insert Table 1 about here 

or ten equivalence classes per half year, with each class labeled in 
terms of a grade -placement (GP) equivalent. The GP of equivalence 
classes was determined by analysis of three- major elementary-school 
mathematics texts (Clark, Beatty, Payne, &. Spooner, 1966; Eicholz &. 



TABLE 1 

Grade Level Spanned by Each "Strand -in 
the Elementary Mathematics: Program 

J 



Strand 


Content. ' • 


Grade level 


TVTTTM 

JWUM 


Number concepts 


1 a 1 n 




Horizontal addition 


i n x n 


IloU 


Horizontal subtraction 


1 • 0- 5* - 




Vertical addition 


. 1.0-5 -y - 


vsu : 


Vertical subtraction 


: .1.5-5. 9 • -J.' 


EQN 


Equations: 


: 1-5-7-9 


MEA 


■ Measurement 


: 1-5-7.9 


HMU 


-Horizontal imulxT&Iicsction 




LAW 


Ijavs of ar±Hime±±c 


5.0-7.9 


VMU 


Vertical multipTTraLxon: 


5.5-7-9 


DIV . 


Division 


5.5-7.9- 


FRA 


Fractions 


5.5-7.9 


DEC 


Decimals 


4.0-7.9 ; , ' 


NEG 


Negative numbers 


6.0-7.9 
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O'Daffer,. 1968; Suppes, 1966), Data collected during several years of 
the earlier drill-and-practice mathematics program at Stanford were 
used to arrange the equivalence classes in an increasing order of 
difficulty and to insure that new skills (e.g., regrouping in sub- 
traction) were introduced at the appropriate point. 

In addition to ordering the equivalence classes within a strand, 
it was necessary to determine how much emphasis to give each strand at 
a given grade level. To determine this emphasis, we divided the curric- 
ulum into Ik parts, each corresponding to a half year. A probability 
distribution was defined for the proportion of problems on each strand 
for each half year. Both the problem count from the three textbook 
series mentioned above and. the average latency for problem types based 
on past data were used to define the curriculum distribution. The final 
proportions in terms of time and problems for each half year for each 
strand are shown in Table 2. 

Insert Table 2 about here 

The analysis contained in Table 2, which embodies not only empirical 
analysis, but also some normative decisions about relative emphasis in 
curriculum, is one of the few explicit quantitative analyses of curriculum 
distribution in elementary-school mathematics to be found anywhere in the 
mathematics education literature. 

A student's progress through the strands structure is purely a 
function of his own performance and is independent of the performance 
of other students; in, fact, his progress on a given strand is independent 
of his own performance on other strands . A scheme defining movement 
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TABLE 2 



Proportion of Time and Proportion of Problems for Each Strand for Each Half Year 



Strand 














Half 


year 














1*0 


l»5 


d 4 U 


2.5 


3.0 


3-5 


)i n 
4 . 0 


)i IT 
4.5 


5.0 


5.5 


D .0 


C IT 
6-5 


7.D 


7-5 


NUM 


PT 


50 


24 


24 


17 


10 


5 


7 


7 


8 


11 


14 


10' 


15 


15 




pp 


.36 


!l8 


16 


12 


10 


4 


8 


8 


10 


14 


20 


10 


19 


19 


HAD 


PT 


26 


21 


21 


9 


14 


9 




















ri 


32 


28 


26 


10 


14 


8 


















HSU 


PT 


Ik 


10 


16 


9 


4 






















PP 

rr 


18 


14 


16 


10 


4 




, 
















VAD 


PT 


10 


10 


9, 


19 


19 


7 


8 


2 


3 


1 












PP 

rr 


Ik 


12 


12 


22 


20 


6 


10 


2 


4 


2 










VSU 


PT 




9 


8 


15 


22 


10 


13 


3 


3 


1 




- 








PP 

rr 




12 


12 


18 


20 


8 


10 


2 


4 


2 










EQN 


PT 




17 


12 


16 


17 


14 


17 


7 


5 


7 


7 


8 


15 


15 




PP 

rr 




10 


10 


12 


16 


12 


20 


■ 8 


8 


12 


12 


10 


19 


19 


MEA 


PT 




9 


10 


j 

8 


8 


n 


7 


7 


5 


5 


5 


5 


5 


5 




pp 
rr 




6 


8 


6 ' 


6 


6 


8 


8 


8 


8 


8 


6 


6 


6 


HMD 


PT 








7 


3 


8 


5 


3 


2 ' 














pp 
rr 








10 


6 


14 


10 


6 


8 












IAW 


PT 










3 


5 


5 


3 


3 


3 


l 


l 


8 


8 




PP 










4 


6 


6 


4 


6 


6 


2 


2 


10 


10 


VMU 


PT 












10 


5 


14 


6 


8 


7 


5 


8 


8 




PP 












14 


6 


16 


8 


4 


4 


2 


4 


4 


DIV 


PT 












15 


22 


34 


48 


33 


40 


13 


14 


14 




PP 












18 


10 


16 


16 


6 


8 


2 


3 


3 


ERA 


PT 












6 


4 


15 


13 


20 


17 


18 


10 


10 




PP 












4 


4 


24 


22 


32 


32 


26 


10 


■10 


DEC 


PT 














7 


5 


4 


11 


7 


36 


10 


10 




PP 














8 


6 


6 


14 


10 


-38 


10 


10 


WEG 


PT 






















2 


4 


15 


15 




PP 






















4 


4 


19 


19 



Note. --FT = proportion of time; PP' = proportion of problems. 



through a strand uses the pattern of correct and incorrect responses to 
insure a rate of movement that reflects performance » The movement scheme 
will "be described in detail elsewhere (Suppes, Searle, & Lortonj in prep- 
aration). The overall determination is based on a model that assumes 
independence of responses for problems of a given equivalence class and 
is defined so that a student with average performance gains- one year's 
GP in one school year of CAI time, which ranges from 6 to 10 minutes per 
school day. A detailed description of the strands curriculum is given 
by Suppes, Goldberg, Kanz, Searle, and Stauffer (l97l)« 

DESIGN OF THE EXPERIMENT 

Equipment 

The central computer processor was the Institute's PDP-10 system 
located on the Stanford campus. On-line, real-time communication was 
maintained with the participating schools located in California, Florida, 
Oklahoma, Texas, and the District of Columbia by means of 'dedicated 
telephone lines* 

The student terminals were KSE Model- 33 teletypewriters. The tele- 
typewriters communicate information to and from the central computer 
system at a rate of about 10 characters per second. All. of the ele- 
mentary mathematics exercises were typed at the terminal under computer 
control, and keyboard responses were given by the students. The details 
of exercise format and student responses are described in Suppes, Jerman, 
and Brian (1967) and Suppes and Morningstar (1972). 
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Students and Number of Sessions 

The students participating in this exxjeriraent included the entire 
population receiving the elementary ma.themat.ics and language arts 
curriculum whose current, average GP on the mathematics strands cur- 
riculum was equal to or greater than 2<h and equal to or less than 5.9, 
and who had taken at least 15 mathematics CA.I sessions,, 

Five levels of intensity for number of CAI sessions were selected- 
Treatment groups 1, 2, 3, ky and 5 were assigned. 10, 30, 70, 100, and 
130 sessions, respectively, and 77 students were randomly assigned to 
each group. 

Session limits were imposed on a calendar basis so that students 
with low numbers of sessions received them distributed throughout the 
experimental period, A participating student had no control over the 
type of lesson, mathematics strands or language arts, he received. 
Whether he signed on for strands or language arts he was given a 
mathematics lesson if he was eligible for one, Otherwise, he 
received a language arts lesson. 

Eligibility for a mathematics session was decided according to 
the following algorithm 0 

TS, 



If NS 



i < [ D i(^) + ' 2 ] aM TS i< TO i 



then student 1 received a mathematics session; otherwise, he received 
a language arts session. In the algorithm, 

NS^ =s -number of sessions taken during the experimental period 

by student i, ■ , 
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ts- = total number of sessions student i was to receive during 

the experimental period, 
TD^ = number of calendar days in the experimental period for . 

student i, 

= number of calendar days elapsed for student i in the 
experimental period, 
and the brackets denote the next greatest integer. For example, suppos 
a student was in a 70-sessions group (TS^ = 70 ) and the experimental 
period was 150 days : (TB^ = 150), If on the 21st day he was in the 
experiment (D^ = 21) he had taken II sessions ■ (KS^ = U)/ then he 
..would have received a mathematics session when he signed on, since 

' ' 'u^ns.^d. (ii) + 2 ].[ a .(^.) + 2 ]- 12 . 

Some students signed on more than once a day in order to obtain the 
assigned number of sessions. 

The actual number of mathematics sessions a student received was 
monitored daily, The assistance of teachers and proctors was sought to 
help students achieve the number of sessions they were assigned. 
Teachers were urged not' to- give compensatory off-line work to those 
students assigned to low numbers of on-line sessions, and, in general, 
not to alter the classroom work of any student because of his partici- 
pation in the experiment. 
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THE MODIFIED STANFORD ACHIEVEMENT TEST 



One aim of the experiment was to develop a reliable and valid test 
of achievement that could be administered 'on line', i,e., by computer, 
at student terminals „ It was hoped that administering the test on line 
would standardize procedures for giving it, that the schools would be 
spared the difficulties of having yet another test to administer, and 
that the clerical task of getting test scores into the computer for 
data analysis would be minimized, The Arithmetic Computation Section 
of the Stanford Achievement Test (SAT) was used as a guide in developing 
a modified SAT (M3AT) that could be administered on line at student 
terminals. 

Description of the SAT 

The Arithmetic Computation sections of three SAT batteries,. 
Primary II, Intermediate I, and Intermediate II, were used as models, 
For all three levels, forms W and X were examined. 

Computation problems on the" Primary II test are worked in the test 
booklet* All questions on the computation sections of the Intermediate 
level tests (I and II) are multiple choice, and responses are entered 
in the test booklet or on a standard answer sheet. The responses are 
letters between a and j« Each question has four or five possible 
responses; the letters a-d and a - £ are used for one question, the 
letters e-h or f-J_ are used for the next question. 

'Several properties of the tests are shown in Table 3. The table 
contains, for each level, the number of problems, the time allowed for 

Insert Table J about here 
Z ih 



TABLE 3 



Characteristics of Computation Section of the GAT 



Test level 


Number of 
problems 


Time 
(min. ) 


Placement 
on form W 


Placement 
on form X 




.25 


.50 


.75 


•25 


.50 .75 


Primary II 


60 


30 


2.k 


5-k 


k.k 


2.2 


3.3 k.k 


Intermediate I 


39 


35 


3.5 


k.6 


5.8 


3-5 


k.~( 6.0 


Intermediate II 


39 


35 


k.k 


6.1 


8.5 


k.6 


6.2 8.7 
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administration, and the normed GP corresponding to error rates of .75, 
♦50 ; and .25. These last figures are shown separately for forms W and X. 

C onst r uction of the MSAT 

Three sets of parallel items were constructed for each item in the 
computation section of the SAT. The method used to construct items de- 
pended on whether the type of SAT 'item being considered appeared in the 
CAI mathematics strands curriculum. 

Each SAT problem of a type that occurs in the CAI curriculum was 
labeled with the equivalence class number into which it fitted. Both 
forms (W and X) were examined and labeled. If similar items for both 
SAT forms fell in the same equivalence class, three items from that 
equivalence class were used; if not, a new class definition was con- 
structed that included the attributes of both classes, and three 
problems were written to fit this definition. 

If a test problem was of a type that did not occur in the strands 
curriculum, a new class definition was written that described the salient 
features of the exemplars from both test forms. Then three items were 
written using the definition. 

A careful analysis was made of the answer choices presented in 
Intermediate I and II. The distractors for each addition problem differ 
from the correct answer in one digit that occurs in any but the" left-most 
position. The distractors for each subtraction problem differ from the 
correct answer in one digit that occurs in any but the left-most position, 
or the left-most digit is one less than that of the correct answer. 
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Two of the distractors for each multiplication problem in the SAT 
differ from each other in one digit and are "both one digit too short. 
Othei* possible wrong answers differ from the correct answer in any but 
the left -most digit. In the case of a two-digit multiplier, one wrong 
answer for the MSAT was constructed by using only one of the digits of 
the multiplier to obtain a product* Distractors for decimal problems 
using dollar signs are the same as those for comparable integer prob- 
lems. In other decimal problems in the SAT, the wrong answers differ 
from the correct answer in the placement of the decimal point- specifi- 
cally, they are chosen from (CA)(lO ), K = ±1, i2 ; where CA is 
the correct answer. 

Division problems in the SAT occur in three formats: A/B, B * A 
and l/A of B. Some choices have a remainder and others do not, 
regardless of the correct-answer remainder. Some remainders are larger 
than the divisor. The same method was used to construct MSAT answer 
choices for all three types of division' problems. Distractors were 
constructed by changing one of the digits of the quotient, and where 
appropriate, choosing a random remainder. 

One dilemma was encountered in constructing answers for division 
problems. • Two problems on the X form of Intermediate I and one on the 
W form have two technically correct answers. The following item exem- 
plifies these problems. 
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A) h2 

B) 50, EEM 8 

c) 52 

D) 5, HEM 8 ' 

E) NOT GIVEN 

Although C is the Correct 1 answer, an argument can be made that B is 
also correct. This answer choice was not used -in constructing wrong 
answers for the M3AT* In both Intermediate I and Intermediate II there 
are four problems for which the correct answer is n not given." 

Administration of the MSAT 

The MSAT was administered by randomly selecting one of the three 
items written for each problem number. Thus, while approximately one- 
third of the items on one MSAT administration would be repeated on a 
second administration, it was unlikely that any two students sitting 
next, to each other would receive the same test. A record was kept of 
the items that comprised each student's test* 

The MSAT was administered as much like the SAT as possible. The 
major exception concerns the possibility of students taking the paper 
and pencil SAT to return to problems already completed or skipped over. 
Students may skip any problem on the MSAT simply by typing the return 
key on the teletype terminal. However, once a student answers or skips 
an MSAT problem there is no way for him to return to it. 

Tests were administered at the terminal, preferably at one sitting. 
If a student did not complete the test in one sitting, he began where 
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he left off at his next sign on. The maximum time allowed to complete 
the test was the time allotted for adminirtratioi: of the computation 
section of the- SAT, plus one-half the time it takes to type the test 
at the terminal. 

Response modes or. the MSAT were as close as possible to those on 
the SAT. For multiple-choice items the student types a single letter. 
For constructed responses the student types the numerical answer as he 
would for an ordinary strands item with the strands error response sup- 
pressed. The student may alter his response using the rubout key. When 
he has completed his answer, he types the return key. If a student 
working a multiple-choice problem types a number instead of a letter, 
he is told to type a letter and given another chance. As soon as the 

1 

return key is typed the program moves to the next problem. There is no 
time limit for individual problems. Many of the problems require pencil 
and paper computation. 

The procedures that differed from the usual mode of responding at 
the teletype were explained to the student, A message at the beginning 
of each test mentions the following; 

1. Use paper and pencil when necessary; 

2„ Answer multiple-choice questions with a letter (for 
Intermediate I and II); 

3- Use the return key after a response (for Intermediate I 
and II); 

k. Use the return key to skip a problem; 

5. Use the rubout key to erase; 

6. Be aware of the time limit for the entire test* 
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These six points and the printed instructions vere explained and ampli- 
fied by the CM proctors before students took the MSAT* The. instructions 
printed for Primary II and for the two intermediate forms of the MSAT 
are displayed in Figure 1. As in the SAT, students were given a 

Insert Figure 1 about here 

non^raded sample problem before beginning the MSAT* Students were not 
given any results at the end of a test session.. A coded number contain- 
ing the score was printed at the. end of the session for use by teachers. 

Choice of level of test* The level of the MSAT administered to 
a student is made with reference to his average GP in the mathematics 
strands curriculum. The choice algorithm is presented in Table *4-* 

Insert Table k about here 

The MSAT was administered in January, 1972, at the beginning of 
the experimental period. and again in May immediately after the experi- 
ment ended. 

DATA ANALYSIS AND RESULTS . 

Analysis of Variance • 

As mentioned earlier, treatment groups 1, 2, 3; ^ and 5 were 
assigned 10, 30, 70, 100, and 1J0 sessions, respectively, and 77 
students were randomly assigned to each of the five treatment groups. 
Complete data were obtained for 60 students in group 1, 62 students 
in group 2, 60 students in group 3, 60 students in group k, and 
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Ins li'UcLj.i.'UC for MSAT Primar y II: 



MODIFIED S. A. T. STRAW D 

THIS IS A SPECIAL TEST. YOU HAVE 33 MINUTES TO WORK 
ON IT. YOU MAY USE PENCIL AND PAPER. 

TYPE THE RETURN KEY TO GET THE NEXT PROBLEM. 

HERE IS A SAMPLE PROBLEM. 

h + 9 = _13_ 



Instructions for MSAT Intermediate I and Intermediate II: 



MODIFIED S. A. T. STRAND 

THIS IS A SPECIAL TEST. YOU. HAVE 38 MINUTES TO WORK 
ON IT. ' USE PENCIL AND PAPER TO WORK THE PROBLEMS . 

ALL QUESTIONS ARE MULTIPLE CHOICE. TYPE A LETTER AND 
THEN THE RETURN KEY. USE THE RUBOUT KEY TO ERASE. USE 
THE RETURN KEY TO SKIP A PROBLEM . 

HERE IS A SAMPLE PROBLEM. 

k + 9 = 

A) 12 

B) 5 

C) 13 

D) k9 

E) NOT GIVEN 
C 



Fig. 1. Instructions printed for the Primary II and for the 
Intermediate forms of the MSAT. 
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TABUS k 

Algorithm for Choosing an MSAT Test Level 



Test Level 


Range of average GP on 
date of administration 


Primary II 


2.0 - 3.5 


Intermediate I 


3.6 - 1*.3 


Intermediate II 


k.9 - 6.5 
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70 students in group 5, This information is summarized in Table. 5* 
^ which also gives the means and standard deviations for number of 

Insert Table 5 about here? 

sessions actually' taken within each of the five treatment groups. 
The number of sessions taken fell veil short of the number assigned 
in groups 3* and 5, primarily because of difficulties in scheduling 
extra CAI sessions in the schools, However, .the groups remained suf- 
ficiently distinct to warrant proceeding with, analysis of variance. 

Analyses of variance were performed taking MSAT scores and 
average GP of the mathematics strands as dependent measures, In 
order to make comparisons across ail three MSAT battery scores, we 
used SAT scales to convert MSAT raw scores to GP scores. Analyses 
of variance were performed on pretreatment measures as well as post- 
treatment measures to check for any bias in the assignment of stu- 
dents to treatment groups* 

The pretreatment analyses of variance for the strands average' GP 
and MSAT scores are presented in Tables 6 and 7* Both F ratios are 

Insert Tables 6 and 7 about here 

small. In addition, the correlation between pretreatment, strands 
average GP and the number of sessions taken was ,013* and the cor- 
relation between MSAT pretest GP and number of sessions taken was 
-.022* It is reasonable to conclude that random assignment of stu- 
dents to treatment groups was essentially achieved*, 
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TABLE 6 

Analysis of Variance for Pretreatment, Mathematics Strands 
Average GP for the Five Treatment Groups 



Treatment group 





1 


2 


3 


k 


5 


Sample size 


60 


62 


6o 


60 


70 


Mean 


Ik 18 


3.88 


3-96 


3-9^ 


k. .12 


Standard deviation 


.87 


.87 ■ 


• 77 


.8k 


•91 



Analysis of variance 





Sum of squares 


df 


Mean square 


F ratio 


Between groups ' 


!*,1113 


k 


1.0278 


lA08* 


Within groups' 


22k. l^k 


307 


«7302 




Total 


228.2687 


311 







*Nonsignif icant 
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TABLE 7 

Analysis of Variance for Pretreatment, MSAT GP Scores 
for the Five Treatment Groups 



Treatment group 





1 


2 


3 


1+ 


5 


Sample size 


6'0 


62 


6c 


60 


70 


Mean 


fc.79 


k.23 




1k6i 


k.52 


Standard deviation 


1.50 


1.31 


1.21 


1.72 


1.6k 



Analysis of variance 





Sum of squares 


df 


Mean square 


F ratio 


Between groups 


10.5828 


k 


2.6^57 


1.185* 


Within groups 


685. 6 kkk 


307 


2.2331* 




Total 


696.2272' 


311 







♦Nonsignificant 
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The posttreatment analyses of variance for the strands average GP 
and M3AT scores are presented in Tables 8 and 9» The F ratio for the 

Insert Tables 8 and 9 about here 

strands GP scores is significant (p < ,01., df = ty307)j and we note 
that the average GP improvement for the 10-sessions group 1 is only «15 
compared with ,96 for group 5« The F ratio for the MSAT scores is non- 
significant^ ■ but the average GP improvement for the 10-sessions group 1 
is A2 compared with ,76 for group 5- 

Test of the Five Models 

Parameters for the five models were generated twice, once using 
mathematics strands average GP as pretreatment and posttreatment achieve- 
ment measures and once using -M8AT GP scores. Models and parameters using 
strands average GP as the achievement measures are presented in TatJL* 10- 

• Insert Table 10 about here 

The linear model with interaction, Model II, accounts for more of the 
variance, in the dependent variable (posttreatment average GP) than does 
any of the other models, but despite the inclusion of a term for the 
interaction of number of sessions with pretreatment GP, it represents 
only a slight improvement over Model I, the simple linear model. Assuming 
N = 120 or slightly less than one session per day for a school year and 
taking a^ ~ „0123 from Model I, we can project - !• - l*k8. That 

is to say, .if a student from this population takes about one strands ses- 
sion per day for an entire school year, we can expect his strands average . 
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TABLE 8 

Analysis of Variance "for Post-treatment, Mathematics Strands 
Average GP for the Five Treatment Groups 



Treatment group 





1 


2 


3 


h 


5 


Sample size 


60 


62 ' 


60 


60 


70 


Mean 


4.33 


h.32 


h.6o 


1+.85 


5.08 


Standard deviation 


.89 


.Qk 


.76 


.9h 


.96 



Analysis of variance 





Sum of squares 


df 


Mean square 


F ratio 


Between groups 


28.3792 


h 


7.09^8 


9.088* 


Within groups 


239.6608 


307 


.7807 




Total 


268.0U00 


311 







*Signif icant, p < .01. 
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TABLE 9 

Analysis of Variance for Posttreetaient, MSAT GP Scores 
for the Five Treatment Groups 



Treatment group 





1 


2 


3 


k 


5 


Sample size 


60 


62 


60 


60 


70 


Mean 


5.21 


^.82 


J+.89 


5.29 ■ 


5.28 


Standard deviation 


1.50 


1.36 


1.39 


I.71 


l.kk 



Analysis of variance 





Sum of squares 


df 


Mean square 


F ratio 


Between groups 


12.9009 


k 


3.2252 


l.h6k* 


Within groups 


676.V771 


307 


2.2035 




Total 


689.3780 


311 







*Nonsignif icant 
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TABLE 10 

Parameters Generated for the Five Models Using Mathematics Strands 
Average GP as Pre treatment and Posttreatment Measures 



ModelJC: Bd^) = a Q + + a^. 

Parameters: a Q = .305, a = .930, a^ s .012. 
Multiple correlation = .95^- 

Model II: E(l' ) = a^ + a_T.- + a^N. + a^T.-N.. 
i2 / 0 1 il 2 l 3 il l 

Parameters: a Q = .027, a 1 = *998, a^ r= .018, a^ t= - .001. 
Multiple correlation = • 955 • 



Model 



III: E(in T.J = a + a n in T + a 0 in N. . 
s i2 y 0 1 il 2 i 



Parameters: a Q = .Okk, a^ = .817, - .010. 
Multiple correlation = . 9I+I. 

Model IV : E(T.J = a_ + a_T._ + a^ in N. + a n (in N.) 2 + a, (in N.) 3 . 
v i2' 0 1 il 2 1 3 V i' 4 V i' 

Parameters: a Qv = .698, a 1 ^ .928, = .036, a^ = .126. 
Multiple correlation = *953. 

Model V: E(in T. 0 ) = a^ + a n N.T. n . 

\2' 0 1 1 il 

Parameters: a^ = * 363^ = .0001. 
Multiple correlation = .632. 



Note.~-T^ = pretreatment strands GP for student i, 
« posttreatment strands GP for student i, 
N. = number of sessions taken hy student i. 
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GP to increase by about a year and a half. Data presented later show 
that strands average GP underestimated both GP measured by paper and pencil 
administrations of the SAT and GP measured by the MSAT. This improvement 
of l.lj-8 can be compared with an expected GP increase over a school year 
of .5 to .k in. the SAT computation subtest for hearing-impaired students 
receiving ordinary instruction (Gentile & DiFrancesca, 1969). 

Models and parameters using MSAT GP as pretreatment and posttreatment 
measures are presented in Table 11. The multiplicative model from econometric 

Insert Table 11 about here 

that assumes weighted interaction of number of sessions with pretreatment 
GP, Model III, accounts for more of the variance in the posttreatment 
measure than does any of the other models, but P as with the strands 
average GP, it represents only a slight improvement over Model I, the 
simple linear model . Again, assuming = 120 and taking a n = .008^4- 
from Model I, we can project T^g - T = 1.01. That is to say, if a 
student from this population takes about one strands session per day for 
a school year of 120 net days, we can expect his MSAT GP to increase by 
about one year. Roughly, we can expect an increase of .1 in MSAT GP for 
every 12 sessions taken. 

Overall, the models using MSAT GP as the achievement measure account 
for about 25 percent less variance in the posttreatment results than do 
those using strands GP as the achievement measure. In the case of Model I, 
this decrease might be due to less contribution from the pretreatment 
scores in accounting for posttreatment scores, less contribution from 
number of sessions, or both, A comparison of the a^ parameter for 
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TABLE 11 

Parameters Generated for the Five Models Using MSA? CP Scores 
as Pretreatment and Posttreatment Measures 



Model I: E{T. n ) = + a_T. n + a_N.. 
v i2 0 1 il 2 i 

Parameters: a Q = 1.116, a 1 = .793, a^ ■- .008. 
Multiple correlation » .811. 

Model II: E(T. 0 ) = a^ + a.T.' + a n N. + aJT.-N. . 
— K i2' 0 1 il 2 i 3 il i 

Parameters: a Q - .939, & 1 " .831, a Q « .012, - .001.* 

Multiple correlation - .812. 

*The F ratio for deletion of a^ is nonsignificant (p < .005") • 

Model III: E(i?n T. n ) = a^ + a n in T. n + a„ Zn N . 
i2' 0 1 il 2 i 

Parameters: a Q = .101, a 1 = .7U, a ? « .006. 
Multiple correlation = .8l8. 

Model IV: E(T. 0 ) ^ a^ + a n T. n + a^ £n N. + a f4n N.) 2 * a } (in N.) 3 . 
• — i2' 0 1 il 2 l 3 i *4 1/ 

Parameters: - a = 2.1+05, a = .784, a = .000, a = - -330, a. = .069; 

u 1 d 3 4 

Multiple correlation - . 8l6. 

Model V: E(in T JO ) = a^ + a N.T. n . 
x i2' 0 1 1 il 

Parameters: a Q = .370, a = .0001. 
Multiple correlation = .^77* 

Note.— = pretreatment MSAT GP for student, i, 
T i2 = posttreatment MSAT GP for student i, 
K. ss number of sessions taken "by student i. 
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Model I, using strands GP (a^ ,==. .930), with Model I, using MEAT GP 
(a 1 = *193)j and a comparison of the a n parameter for Model I, using 
strands GP (a n = .0123), with Model fusing MSAT GP (a n = .008 J i), 
imply that the pretreatment measure and the number of sessions both 
contribute less when Model I uses MSAT GP as the achievement measure 
than it does using strands average GP, a result- that is not at all 
surprising. 

Because of the interest in the contribution of number of sessions 
taken to the GP gain measured by the MSAT, we computed the. 95 percent 
confidence level for coefficient a 0 for Model I (Table 11). The 

d 

result is a^ = .008^ ± -0032, Consequently, the change in GP with 
120 sessions should be between ,62k and 1.392, admittedly a large interval. 
Assuming that the effect of 120 sessions is somewhat sublinear, we should 
conservatively expect an increase in GP of about „5 that would be due to 
the 120 strands CAI sessions, This conservatively estimated GP gain is 
still superior to the results for ordinary instruction of hearing- impaired 
students already cited* As is evident from Table 10, a larger gain in GP 
measured by -the strands curriculum itself would be anticipated. 

Comparison of SAT, MSAT, and Strands GP 

In order to generalize the MSAT results, comparisons of the achieve- 
ment measures used in this study with each other and with standardized 
tests are needed. Because neither the MSAT nor GP measured by the strands 
curriculum is a common measure, it was decided to estimate the concurrent 
validity of the MSAT GP and strands average GP by comparing them with each 
other and with paper and pencil administrations of the SAT. 
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Sixty students were drawn at random from among participating stu- 
dents in three of the residential schools.. Selection of the students 
.was stratified so that h were chosen for each of the 15 cells arising 
from the three MSAT forms (Primary II, Intermediate I, Intermediate II) 
and five treatment groups. Two of the- k students were chosen at random 
and assigned to group I; the remaining 2 were assigned to group II . There 
were then 30 students (2 from each form-by- treatment cell times 15 cells) 
assigned to group I and 3^ assigned to group II. Group I received paper 
and pencil administration of the SAT Arithmetic Computation Subtest 
(SAT-COMP), form W, before receiving the pretreatment MSAT^ and group II 
received the SAT-COMP after receiving the pretreatment MSAT* The roles 
of groups I and II were reversed for the posttreatment measure. Group II 
received the SAT-COMP before the posttreatment MSAT, and group I received 
the SAT-COMP after the posttreatment MSAT* Pretreatment and posttreatment 
strands GP scores were also -recorded. 

Complete data were obtained for kk of these students, The loss of 
l6 was solely due to such random factors as student illness, change of 
schools, and administrative errors „ Means and standard deviations ob- 
tained by the kk students for pretreatment and posttreatment SAT GP, 
MSAT GP, and strands GP are displayed in Table 12, It should be noted 

Insert Table 12 about here 

from Table 12 that the SAT consistently' gave the highest estimate of GP 
for this group of students, the MSAT consistently gave the second highest 
GP estimate, and the strands GP consistently gave the lowest GP estimate, 
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Evidently, both the MSAT and the strands average GP measures underestimated 
GP measured by paper and pencil administration of the SAT* 

A matrix of simple correlations for the GF scores obtained "by the 
kk students on pretreatment and posttreatment SAT, MSAT, and. mathematics 
strands is given in Table 13- These correlations are fairly large, but 

Insert Table 13 about here 

they are not sufficiently large to identify SAT GP, MSAT GP, and strands 
■ GP as parallel measures. According to Lord and Novick (1968), two dis- 
tinct measurements X and X, are parallel if for every subject s 

gs hs 

in the population, jt = rt, and a(E ) = a(K ), where tt indicates 
* gs hs gs 7 v hs " 

measurements with the same true scores but possibly different error vari- 
ances. More intuitively, two measurements are parallel if their expecta- 
tions are equivalent and their observed score variances are equal* This 
is not true of the three GP measur.es „ 

D ifference Scores 

Table 1^ displays a matrix of simple correlation coefficients obtained 
from number of sessions, pretreatment and posttreatment MSAT GP, and 

Insert Table Ik about here 

pretreatment and posttreatment strands GP for. the 312 students who partici- 
pated in the experiment,, Table Ik also includes correlation coefficients 
for the difference scores, MSAT A GP (posttreatment minus pretreatment 
MSAT GP) and strands A GP (posttreatment minus pretreatment strands ' 
average GP ) * 
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The data reported in Ta.b3.es 13 and Ik allow comparisons of the 
^-student sample with the full sample of 312* The correlation of the 
strands pretreatment GP and the strands posttreatment GP is ,860 :in the 

student sample compared with ,863 in the 312-student sample. The largest 
difference in correlation was obtained for the pretreatment MSAT GP with 
the posttreatment strands GF—the student sample correlation being ,711- 
compared with -796 ^or the 312-student sample- As might be expected, the 
correlations for the 312-student sample are in every case higher than 
their counterparts in the student sample* Given these small differ- 
ences in correlation coefficients, it seems reasonable to conclude that, 
with respect to the measures taiten^ the kk- student sample is representa- 
tive of the full 312- student population, 

The anticipated increase in correlation occurs when difference 
scores replace posttreatment scores as the dependent measures in regres- 
sions using number of sessions as the independent variables. The correla- 
tion for sessions arid MSAT posttreatment GP is only .155 compared with a 
correlation of ,273 for sessions and MSAT A GP 0 More striking, but also 
expected, is the correlation for sessions and strands posttreatment GP 
of *kl6 compared with -796 for sessions and strands A GP. 

Means and standard deviations for GP change (posttreatment GP minus 
pretreatment GP) obtained, by the five treatment groups are displayed in 
Table 15* With the exception of average GP change for group 3> a fairly 

Insert Ta,ble 15 about here 

steady increase in GP change with increasing number of sessions is evident 
in the data. It should be noted that the GP change for group 5 whose 
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TABLE 15 

Sample Size, Means, and Standard Deviations of GP Change (Post treatment GP 
Minus Pretreatment GP) Measured by Strands Average GP and MSAT 
for the Five Treatment Groups 











Treatment g 


roup 




1 


2 


3 ■ 


k 


5 


Sample size 






. 60 


62 


60 


■ 60 


?o 




Mean 




.15 


M 


.6k 


.91 


.96 


Strands A GP 


















Standard 


deviation 


.01 


.03 


.03 


.05 


.05 




Mean 




'.k2 


.58 


.48 


.68 


.76 


MSAT AGP 


















Standard 


deviation 


,10 


»09 


.08 


.10 


.11 
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members averaged 75-8^ sessions is .96 for strands average GP and .Jo for 
MSAT GP, Both of these measures underestimated GP measured by paper and 
pencil administration of the SAT to the kk- student sample. Using the 
smaller measure, the group 5 students achieved an increase in mathematics 
computation GP of .J6 during the experimental period of approximately 
five months- This improvement is about double the GP gain indicated by 
Gentile and DiFrancesca (1969) for hearing- impaired students after a full 
school year of traditional classroom instruction* 

SUMMARY AM) CONCLUSIONS 

From the analyses of data given above, we conclude that the mathe- 
matics strands CAI curriculum can lead to substantial increases in mathe- 
matics computation GP when used by hearing-impaired students. The increases 
are sufficient to bring the students to GP gains expected of normal-hearing 
students. Moreover, these gains can be achieved by students working intensely 
for only a few minutes a day in a supplementary drill-and-practice program. 
The actual time spent at a computer terminal by each student ranged from 
6 to 10 minutes for each session,, 

In addition, a simple linear model of student achievement .gives a 
good account of the posttreatment distribution of GP measured either by 
the MSAT or by the strands GP. The investigation of other models, includ- 
ing models with interaction terms, did not lead to any substantial improve- 
ment in accounting for posttreatment GP variance* The results of the 
analysis, including the application of the linear model, indicate that " . 
greater numbers of CAI sessions are beneficial for all students, across , 

hi 



all levels of pretreatraent achievement, Further investigation using 
demographic and other psychological variables vould be desirable in pro- 
viding a more detailed analysis of the extent to vhich posttreatment 
distribution of GP is primarily affected only by pretreatment distribu- 
tion of GP and amount of time at computer terminals „ 

The inequality- averting properties of the mathematics strands cur- 
riculum used as compensatory education for disadvantaged hearing students 
has already been noted by Suppes and Morningstar ( 1970b) and by Jamison, 
Fletcher, Suppes, and Atkinson (1973)* These properties seem to obtain 
to about the same degree for the population of hearing- impaired students 
used in this study, 
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